Track upstream torchtitan changes up to 20260713 by hann-wang · Pull Request #35 · AMD-AGI/ALTO

hann-wang · 2026-07-15T02:14:40Z

The major changes in upstream torchtitan is that mixed-precision is now handled by fsdp instead of autocast. This means that we need to deal with DTensor now even with one GPU only.

Replaces #10.

fix: decomposed linear swap

Copilot

Pull request overview

This PR updates ALTO to track upstream torchtitan changes through 2026-07-13, especially the shift to FSDP-managed mixed precision (and the resulting need to handle DTensor even on single GPU), plus associated config/script updates.

Changes:

Added/adjusted DTensor handling in observers and sparsification (notably WANDA) to work with upstream FSDP mixed-precision behavior.
Updated modifier lifecycle to support config-time conversion (convert_config / on_convert_config) and refreshed several modifier/model config patterns (e.g., ignore lm_head).
Updated example/integration scripts and model registries for upstream API changes; bumped dev version and changelog.

Reviewed changes

Copilot reviewed 58 out of 58 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
version.txt	Bumps project dev version to `0.1.0-dev0`.
CHANGELOG.md	Adds v0.1.0 dev notes (FSDP mixed precision, Instella config removal).
tests/integration/instella_3b_opt.sh	Removes Instella 3B integration script.
tests/integration/gpt_oss_debugmodel_lpt.sh	Adds expert parallel degree flag in integration run.
examples/run.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/WandaStructured_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/Wanda_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/SparseGPT_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/RTN_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/OBS_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/Magnitude_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/llama3.2_1b_mx9.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path (placement adjusted).
examples/GPTQ_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/CosineSimilarity_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/AWQ_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/ALPS_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/AdmmStructured_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
examples/ADMM_Llama_3.1_8B.sh	Sets `HSA_NO_SCRATCH_RECLAIM=1` for torchrun path.
alto/utils/exportation/export.py	Removes tied-embedding hotfix during HF export conversion.
alto/train.py	Updates forward path to align with upstream PP schedule API and FSDP MP expectations.
alto/observers/per_channel_norm.py	Introduces DTensor-aware stats initialization and local handling.
alto/nn/decomposed_linear.py	Converts `DecomposedLinear` to torchtitan `Module` + nested `Config`.
alto/modifiers/sparsification/wanda.py	Adds DTensor support in WANDA sparsification (redistribute/to_local/from_local).
alto/modifiers/sparsification/base.py	Adds `on_convert_config` hook to sparsification base lifecycle.
alto/modifiers/quantization/base.py	Moves quantization init/validation into `on_convert`; adds config-convert hook.
alto/modifiers/pruning/base.py	Adds `on_convert_config` hook to pruning base lifecycle.
alto/modifiers/lpt/base.py	Updates SDPA wrapper integration and adds config-time conversion for Linear→DecomposedLinear.
alto/modifiers/distillation/base.py	Updates optimizer configuration to param-group style; adds config-convert hook.
alto/modifiers/base.py	Introduces `convert_config` + abstract `on_convert_config` for modifiers.
alto/models/patcher.py	Updates rotary-embedding patching to match upstream ComplexRoPE API.
alto/models/llama3/configs/wanda_structured_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/wanda_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/sparsegpt_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/rtn_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/recipe.yaml	Updates commented and active ignore list references to `lm_head`.
alto/models/llama3/configs/obs_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/mx9_wa_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/mx6_wa_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/magnitude_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/lpt_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/guanchen_recipe.yaml	Updates ignore list references to `lm_head` (including commented blocks).
alto/models/llama3/configs/gptq_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/cosine_similarity_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/awq_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/awq_debug_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/alps_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/admm_structured_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/configs/admm_recipe.yaml	Switches ignore list from `output` to `lm_head`.
alto/models/llama3/config_registry.py	Updates trainer config fields for upstream API changes (profiler, optimizer factory, activation checkpointing).
alto/models/gpt_oss/configs/lpt_recipe.yaml	Updates ignore list to `lm_head` and keeps router gate ignore.
alto/models/gpt_oss/config_registry.py	Updates profiler + activation checkpoint fields for upstream API changes.
alto/models/deepseek_v3/configs/lpt_recipe.yaml	Updates ignore list to `lm_head` and keeps router gate ignore.
alto/models/deepseek_v3/config_registry.py	Updates profiler + activation checkpoint fields; removes selective AC option.
alto/kernels/dispatch/conversion.py	Adjusts bias-parameter skip condition while swapping wrapped parameters.
alto/kernels/dispatch/attention.py	Replaces SDPA wrapper with `LPScaledDotProductAttention` matching upstream SDPA class/signature.
alto/kernels/dispatch/init.py	Updates exports/imports for renamed SDPA wrapper class.
alto/components/converter.py	Adds `convert_config` pass to apply modifier config conversions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 59 out of 59 changed files in this pull request and generated 3 comments.

+                        new_config = DecomposedLinear.Config(
+                            in_features=config.in_features,
+                            out_features=config.out_features,
+                            bias=config.bias,
+                            param_init=config.param_init | DecomposedLinear._EXTRA_INIT,
+                        )
+                        setattr(parent, attr, new_config)


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…ALTO into han/torchtitan-20260713

Copilot

Pull request overview

Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.

+                        new_config = DecomposedLinear.Config(
+                            in_features=config.in_features,
+                            out_features=config.out_features,
+                            bias=config.bias,
+                            param_init=config.param_init | DecomposedLinear._EXTRA_INIT,
+                        )


+- Changed
+  - Mixed precision is now handled by FSDP even if world_size=1.
+  - Dropped Instella-3B model config.


Copilot

Pull request overview

Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.

…apper applied

Copilot

Pull request overview

Copilot reviewed 60 out of 60 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

alto/modifiers/lpt/base.py:225

In on_convert_config(), lora_rank is pulled from config.lora_rank, but config here is a Linear.Config produced by torchtitan. This is inconsistent with the runtime conversion path (which uses self.lora_rank) and will likely raise an AttributeError or silently use the wrong rank. Use the modifier-level self.lora_rank when constructing DecomposedLinear.Config.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 60 out of 60 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

alto/modifiers/quantization/base.py:126

on_convert() unconditionally writes to mod._param_init for modules that have quantization_scheme. Target modules are often plain torch.nn.Modules (e.g., torch.nn.Linear) which do not define _param_init, so this will raise AttributeError during conversion.

    def on_initialize(self, model_parts: list[Module], **kwargs) -> bool:
-        if not QuantizationMixin.has_config(self):
-            raise ValueError("QuantizationModifier requires that quantization fields be specified")
-        for m in model_parts:
-            QuantizationMixin.initialize_quantization(self, m)
-
        if self.sequential:
            self._build_sequential_blocks(model_parts)
        return True


        *,
+        attention_masks: None = None,
        scale: float | None = None,


Copilot

Pull request overview

Copilot reviewed 60 out of 60 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

CHANGELOG.md:7

CHANGELOG entry says "Dropped Instella-3B model config", but Instella configs (instella_3b/instella_3b_opt/instella_3b_lpt) are still present in alto/models/llama3/config_registry.py. Either remove those configs or update the changelog to reflect what was actually removed (e.g., only the integration script).

  - Mixed precision is now handled by FSDP even if world_size=1.
  - Dropped Instella-3B model config.

    def on_convert(self, model: Module, **kwargs) -> bool:
        for scheme_obj, targets in self.resolved_config.items():
            for name, module in match_named_modules(model, targets, self.ignore):
                if isinstance(module, BaseAttention):


+        if self.lora_rank > 0:
+            resolved_targets = list(self.resolved_config.values())
+            for _fqn, config, parent, attr in model_config.traverse(Linear.Config):
+                for target in resolved_targets:
+                    if is_match(_fqn, "Linear", target, self.ignore):


hann-wang added 4 commits July 14, 2026 03:48

feat: adapt torchtitan dev 20260713

9f19b2c

fix: apply_rotary_emb_complex patch

3a46ed3

fix: token dispatcher with paded tokens;

b9786c9

fix: decomposed linear swap

feat: basic DTensor support in wanda

a9efbdb

Copilot AI review requested due to automatic review settings July 15, 2026 02:14

hann-wang mentioned this pull request Jul 15, 2026

Initial FSDP support in Quantization/Sparsification #10

Closed

Copilot started reviewing on behalf of hann-wang July 15, 2026 02:15 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread alto/nn/decomposed_linear.py

Comment thread alto/observers/per_channel_norm.py

Copilot started work on behalf of hann-wang July 15, 2026 02:34 View session

fix: restore from_linear() classmethod and update test constructor usage

ff64beb

Copilot AI review requested due to automatic review settings July 15, 2026 02:37

hann-wang removed the request for review from Copilot July 15, 2026 02:37

Copilot finished work on behalf of hann-wang July 15, 2026 02:37

Copilot started work on behalf of hann-wang July 15, 2026 02:42 View session

Copilot finished work on behalf of hann-wang July 15, 2026 02:44

fix: gpt_oss_20b optimizer and dataloader issues

06e4f2f

Copilot AI review requested due to automatic review settings July 17, 2026 06:46

Copilot started reviewing on behalf of hann-wang July 17, 2026 06:47 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Potential fix for pull request finding

e84bc2d

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 17, 2026 06:54

Copilot started reviewing on behalf of hann-wang July 17, 2026 06:55 View session

hann-wang added 2 commits July 17, 2026 01:56

fix: propagate lora_rank to DecomposedLinear.Config

a1a8fab

Merge branch 'han/torchtitan-20260713' of https://github.com/AMD-AGI/…

e45f460

…ALTO into han/torchtitan-20260713

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 17, 2026 07:01

Copilot started reviewing on behalf of hann-wang July 17, 2026 07:02 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread alto/modifiers/lpt/base.py Outdated

Comment thread alto/models/patcher.py Outdated

fix: checkpoint manager failed to access untyped storage if tensor wr…

536c13d

…apper applied

Copilot AI review requested due to automatic review settings July 20, 2026 02:31

Copilot started reviewing on behalf of hann-wang July 20, 2026 02:32 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Comment thread alto/modifiers/quantization/base.py

Potential fix for pull request finding

db5fcf3

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 20, 2026 06:42

Copilot started reviewing on behalf of hann-wang July 20, 2026 06:43 View session

Potential fix for pull request finding

dfc194a

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 20, 2026 06:48

Copilot started reviewing on behalf of hann-wang July 20, 2026 06:49 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Track upstream torchtitan changes up to 20260713#35

Track upstream torchtitan changes up to 20260713#35
hann-wang wants to merge 12 commits into
mainfrom
han/torchtitan-20260713

hann-wang commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

hann-wang commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants